1. Field of the Invention
The embodiments herein relate to management of shared buffer resources in multi-processor computer systems, and more specifically, to a system and method for dynamically limiting the number of outstanding requests for read data a requesting processor or bus agent may accumulate based on the current function it is executing, instead of based on local buffer space available or a fixed allocation.
2. Description of the Related Art
In computer systems and networks, buffers are a convenient means of storing commands, requests and data in transit from one location to another. Buffers are used in a variety of applications, including handling flow control for SMP and switch fabric systems. Buffers may also be used in memory and data allocation, such as a read/write request buffer to manage read and write access operations between multiple sources. Common drawbacks of systems utilizing shared buffers space is contention among several bus agents for the same shared resource and attending congestion issues as well as domination of shared buffers by a high bandwidth process.
In computer systems employing multiple processor cores, it is advantageous to employ hardware accelerator coprocessors to meet throughput requirements for specific applications. Coprocessors utilized for hardware acceleration transfer address and data block information via a bridge. The main system bus then connects the bridge to other nodes that are connected to a main memory and individual processor cores that typically have local dedicated cache memories.
Hardware accelerator engines or coprocessors utilized in multi-processor systems have bandwidth requirements associated with the particular function being executed. Accordingly, the number of memory access operations of one hardware accelerator engine may be substantially greater or less than the coprocessing function of a neighboring hardware accelerator engine necessitating a commensurately larger or lesser use of available buffer address space.
To prevent data loss, a multiprocessor system should not transfer data, commands or requests to a buffer when no space is available because the data will likely need to be resent thereby further taxing system bandwidth. Access to buffer address space must also be allocated fairly to ensure a lower bandwidth process is not completely ignored. Equal access is usually unnecessary because some processes will be able to reliably execute with much lower access than a high bandwidth process. Priority access can be assigned to higher bandwidth processes, but the system still needs a way to prevent total dominance of one processor or bus agent over buffer resources so that processes with lower bandwidth requirements will not stall.
Prior solutions to the problem of efficient buffer allocation include dedicated partitioning of buffers to a specific application or process. For example, if a buffer has sixteen wordlines and is shared among four data channels or applications, each channel might be allocated four buffer wordlines. While fair, this allocation scheme may be inefficient for bursty traffic from a single channel because the high bandwidth channel would not be able to use other buffer locations, even if idle, due to the fixed nature of the allocation.
Another solution to buffer allocation is stop and go commands issued by the buffer to the data channels it services. The buffer manages its resources by signaling when new data may be accepted or when space is reduced. During periods of low volume traffic, each channel receives a “go” signal indicating that space is available and a “stop” signal when space becomes limited to anticipate an overflow condition. This approach offers better utilization of overall buffer space because users are not limited to a fixed allocation of space. However, there is risk with this approach too, as the latency between when the buffer sends out a stop command and the user receives and responds to the command, may present a situation where data is still being sent to a buffer near peak capacity, thereby resulting in an overflow and loss of data. To prevent this situation, stop commands are sent well in advance of the buffer filling to capacity. Consequently, if all senders are bursting data to the buffer, the stop command will be timed to prevent data from being sent to the buffers prior to reaching full capacity. The downside of sending early stop commands is the maximum capacity of the buffers will not be utilized during peak traffic volumes, but an overflow condition is nonetheless avoided.
In any distributed or multi-processor computer system, data and address paths must be provided by a system bus to interconnect the various components of the system. Buses may be internal to a single IC or external board-based interconnect structures. The volume of data traversing the bus as well as the physical length of the bus and the types of devices connected will present a number of design constraints. Because different types of devices have varying levels of throughput capability and latency, the bus must provide a mechanism for flow control to prevent devices from being overwhelmed if not ready to receive data or incapable of receiving data at the rate sent.
A known solution to flow control is a credit based approach in which a sending device receives an allocation of credits to transmit data on the bus. The allotment of credits may represent the capacity of the buffer at a receiving device. When a sender transmits data on the bus its allocation of credits is decremented. When the receiver restores the capacity to receive additional data, it returns the credit to the sender. This approach eliminates the problem of potentially having data in flight to a receiver for which the buffer does not have sufficient capacity and allows the sender to transmit data immediately rather than waiting for permission from the receiver. If the sender exhausts its supply of credits it is temporarily disabled from sending data on the bus.
Credit based flow control requires the sender to maintain a local record of the number of credits available, which is continuously updated as data is sent out on the bus. Credits are typically returned by the receiver as a single pulse or by periodic updates from the receiver. At times a discrepancy may arise between the local record maintained by the sender and actual credits available at the receiver due to conflicts with other bus agents, which is described as a “credit loss.”
The bus agents of a multiprocessor system may be distributed over physically remote subsystems and nodes that are interconnected by a switch fabric. Such modular systems may be further configured to accommodate or compensate for the distributed nature of shared memory. Symmetric Multiprocessing (SMP) systems often require the bus agents to pass data packets to the different nodes of the switch fabric. To increase the efficiency of the system, data packets are grouped together and mapped to virtual channels, which requires extensive use of buffers to hold various elements of disparate bus transactions.
Virtual channels must have independent flow control constraints for bus transactions sharing common interface and buffering resources and are therefore used to separate traffic over a common set of physical links.
Accordingly, there exists a need in the art to overcome the deficiencies and limitations described hereinabove.